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CLAIM: 

l.J A method foY multicasting data messages to members of a multicast group, the multicast 
group comprising a sequencer, one or more clients, one or more data servers, and one or 
more commit sYrvers, the method comprising the steps of: 

transmitlrhg a first data message to the members of the multicast group; 
each data server that receives the first data message requesting the sequencer to 
assign a firstwequence number to the first data message, the first sequence number 
being from a\ sequence of numbers allocated to the data messages, said first 
sequence number following all sequence numbers assigned prior to assignment of 
the first sequence number; 

assigning the first sequence number to the first data message, in response 
to the sequencer receiving a first quantity of the requests to assign a first sequence 
number to the first dara message; 

notifying the qpmmit servers of the assignment of the first sequence 
number to the first data message; 

each of the commit servers sending to the sequencer an acknowledgment 
of the notification of the assignment of the first sequence number to the first data 
message, in response to beW notified of the assignment of the first sequence 
number to the first data message; 
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committing the first sequence number to the first data message, in 
resporise to the sequencer receiving a second quantity of the acknowledgments of 
the notification of the assignment of the first sequence number to the first data 
message; and 

infoVrnng the members of the multicast group of the commitment of the 
first sequenceVumber to the first data message. 



2. A method according to Claim 1, wherein 

said step of each data server that receives the first data message requesting 
10 the sequencer to assigh a first sequence number to the first data message includes 

the step of sending, from said each data server that receives the first data message 
to the sequencer, a data report message identifying the first data message; 

said step of notifying the commit servers of the assignment of the first 
sequence number includes the step of submitting to the commit servers a commit 
1 5 submit message identifying theVirst data message; 

said step of sending t\ the sequencer an acknowledgment of the 
notification of the assignment of tne first sequence number includes the step of 
sending to the sequencer a commit acknowledge message identifying the first data 
message; and 
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said step of informing the members of the multicast group of the 
commitment of the first sequence number includes the step of sending a commit 
message identifying the first data message to the members of the multicast group. 
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3. A method according to Claim 2, 

further comprising the step of transmitting a second data message to the 
members of theVnulticast group; 

wherein said step of sending, from said each data server that receives the 
first data message tb the sequencer, a data report message identifying the first data 
message further includes the step of a first data server sending a first data report 
message identifying tHte first data message to the sequencer after said first data 
server receives the secWd data message, said first data report message also 
identifying the second data message. 

4. A method according to Claim 2, furtheAcomprising the steps of: 

transmitting a second data message to the members of the multicast group; 
each data server that receives the second data message requesting the 
sequencer to assign a second sequence number, the second sequence number 
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b^rig from the sequence of numbers allocated to the data messages, said second 
seqirtfnee number following all sequence numbers assigned prior to assignment of 
the second sequence number, to the second data message, said step of each data 
server thVt receives the second data message requesting the sequencer to assign a 
second sequence number to the second data message, includes the step of sending 
from said eac\ data server that receives the second data message to the sequencer 
a data report message identifying the second data message; 

assigning the second sequence number to the second data message, in 
response to the sequencer receiving a third quantity of the requests to assign a 
second sequence number to the second data message; 

wherein said step of notifying the commit servers of the assignment of the first 
sequence number further included the step of notifying the commit servers of the 
assignment of the second sequence number, said commit submit message identifying the 
first data message also identifying the second data message, 

A method according to Claim 2, wherein the members of the multicast group 
deliver the data messages to their rcspectrVe upper layer applications in order of 
progressing sequence numbers, further including the step of using a receiver driven, 
negative acknowledgment-based approach to imp^pve reliability of delivery of the data 
messages. 
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6. \A method as in any one of Claims 1-5, wherein said data servers store said data messages 
transmitted to the multicast group, the multicast group further comprising checkpoint 
servers, the method further including the steps of: 
step for message consolidation; 
^tep for garbage collection; and 

steb for storing said first sequence number in stable storage. 
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A method for processing data messages multicast to members of a multicast group, the 
multicast group comprising^ sequencer, one or more clients, one or more data servers, 
and one or more commit servers, the method comprising the steps of: 

each data server that receives said each data message requesting the 
sequencer to assign a sequence number, from a sequence of numbers allocated to 
the data messages, to said each d^ta message, in response to receiving each data 
message; 

assigning a sequence number Yellowing all sequence numbers assigned 
prior to assignment of the sequence number to said each data message, in 
response to the sequencer receiving a fir^t quantity of requests to assign a 
sequence number to said each data message; 

notifying the commit servers of each\assignment, each notification 
identifying said each assignment by said each data v message and the sequence 
number assigned to said each data message; 
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each of the commit servers sending to the sequencer an acknowledgment 
of said each notification, in response to being notified of said each assignment, 
said acknowledgment identifying said each data message; 

\ committing said each assignment, in response to the sequencer receiving a 
second quantity of the acknowledgments identifying said each data message; and 
Worming the members of the multicast group of each commitment. 

8. A method according taClaim 7, wherein 

the meVibers of the multicast group deliver the data messages to their 
1 0 respective upper \aycr applications in order of progressing sequence numbers; 

said data servers store said data messages transmitted to the multicast 
group; \ 

further including the step of using a receiver driven, negative 
acknowledgment-based approach to improve reliability of delivery of the data 
15 messages. \ 

iid each data message is associated with a unique 
associated message ID, the step of using further 




9. A method according to Claim 8, wherein j 
message ID and is identifiable from its 
includes the steps of: 
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each member of the multicast group identifying gaps in a progression of 
sequence numbers known by said each member of the multicast group to have 
been Committed to data messages received by said each member of the multicast 
group; 

if &id each member of the multicast group does not know a first message 
ID, said firs\ message ID being associated with a first data message, a first 
sequence number within one of said gaps having been previously committed to 
said first data me&age, said each member of the multicast group querying one of 
said commit servers \p obtain said first message ID; and 

if said each meViber of the multicast group has not received said first data 
message, querying one oX said data servers to retrieve said first data message. 



10. A method according to Claim 8 9 wherein said each data message is associated with a 

unique message ID and is identifiable\from its associated message Tt>, the step of using 
1 5 further includes the steps of: 

each member of the multicast group identifying gaps in a progression of 
sequence numbers known by said each member of the multicast group to have 
been committed to data messages received by said each member of the multicast 
group; 

2 0 said each member of the multicast ^roup querying one of said data servers 

to retrieve said first data message. 
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A method according to Claim 10, further comprising the step of said sequencer 
periodically generating and sending heartbeat messages to the members of the multicast 
group, each said heartbeat message containing an associated largest sequence number, 
said associated largest sequence number being the last sequence number committed at a 
time substantially equal to a time said heartbeat message is generated. 



12. A method according to Claim 8, further comprising the step for periodic message 
consolidation. 



10 13. 
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A method according ro Claim 8, wherein the multicast group further comprises one or 
more checkpoint servers, the method further comprising the step of performing periodic 
message consolidation by said checkpoint servers at message intervals determined 
through a common consensusX protocol, each message consolidation producing a 
checkpoint associated with said e<rch message consolidation, said checkpoint associated 
with said each message consolidation corresponding to a terminal data message, said 
checkpoint associated with said eacrn message consolidation containing checkpoint 
information, the checkpoint information b&ng sufficient for a first upper layer application 
of said upper layer applications to reconstruVt a cumulative system state said first upper 
layer application would attain upon receivin^said terminal message and all said data 
messages that preceded said terminal message. 
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i A method according to Claim 13, further comprising the step of said checkpoint 



se: 



periodically generating and sending checkpoint reports to said sequencer, each 
checkpoint report corresponding to latest checkpoint at the time said each checkpoint 
report is Venerated, said each Checkpoint Report identifying a sequence number of its 
corresponding terminal data message, said each checkpoint report carrying size data of 
the latest checkpoint 
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A method according to Claim 14, further comprising step for synchronizing a first 
asynchronous upper layer process of a first asynchronous member of the multicast group 
with other members of th\ multicast group, said first asynchronous member not being 
said sequencer or one of said oata or commit servers. 
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16. A method according to Claim 1 4 further comprising the step of synchronizing a first 
asynchronous upper layer process of a tirst asynchronous member of the multicast group 
with other members of the multicast group, said first asynchronous member not being 
said sequencer or one of said data or commit servers, said synchronizing step including 
the steps of: 

said first asynchronous member r^rieving a first checkpoint from said 
checkpoint servers; 

said first asynchronous process retrieving all committed data messages 
following terminal data message corresponding to Yhe first checkpoint; 
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delivering said first checkpoint to said first asynchronous upper level 
process; 

delivering said all committed data messages following the terminal data 
menage corresponding to the first checkpoint to said first asynchronous upper 
level process; and 

sai<i first asynchronous upper level process processing said delivered 
checkpoint and said delivered data messages to achieve a system state identical to 
system states ofWher members of the multicast group. 



10 17, A method according t& Claim 8, wherein said each data message bears a 

corresponding logical timestamp, said logical timestamp including a most recent 
sequence number known to original sender of said each data message when said each 
data message was first sent. 

15 18. A method according to Claim 16,Vvherein said each data message bears a 
corresponding logical timestamp, said logical timestamp including a most recent 
sequence number known to original sender on^aid each data message when said each 
data message was first sent. 

20 19. A method according to Claim 1 8, further comprising\he step of: 
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the data servers deleting said stored messages that have logical 
checkjJbints older by a maximum logical lifetime number at the time of deletion 
than a mo\t recent sequence number known at the time of deletion. 



20. A method accordingSto Claim 14, further comprising the step of: 

said data severs deleting the stored data messages that are older than the 
latest checkpoint. 



2L A method according to Claim 16V wherein the multicast group further includes stable 
10 storage wrileable by said sequencer, said method further comprising the step of said sequencer 
storing in said stable storage said assigned sWience number before said step of notifying the 
commit servers. 
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